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ABSTRACT 



To explicate the factors behind standards implementation, 
the results of a project to develop and pilot test a process that bridges the 
gap between setting standards and improving student achievement is presented. 
The text focuses on the Implementing Academic Standards (IAS) program, which 
was designed to develop and pilot-test a process for implementing Minnesota's 
high standards for student learning. The paper opens with an overview of 
theory and research on standards-based reforms, focusing on types of 
standards, definitions of standards-based reform, and reform implementation. 
To illustrate how the process works, background information on Minnesota's 
standards movement is provided. The bulk of the text is devoted to the 
implementation and the effects of IAS. Details on research questions, 
data-collection methods, and findings are presented. A summary of the results 
of the study and implications for further efforts to implement 
standards-based reform are given. Areas where further research is needed to 
increase understanding of the change theory behind standards-based reform are 
likewise identified. (RJM) 
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Standards are the predominant theme in current thinking about educational reform. 
Numerous groups, from discipline-based organizations to state department of education to 
individual school and school districts have developed standards. Yet, standards by 
themselves will not increase student achievement. Schools need to undergo a complex series 
of changes before standards can have an impact on student learning. This process includes 
not only modifications of curricula, but also giving teachers the time and resources to 
incorporate the standards into their instructional practice and beliefs. 

This paper presents the results of a project to develop and pilot test a process to bridge the 
gap between setting standards and improving student achievement. The study has 
implications for any group trying to improve student achievement through standards-based 
reform. If educational resources are not to be wasted, we need to increase our knowledge of 
how to integrate standards into daily classroom practice. 



Theory and Research on Standards-Based Reforms 

Standards are one type of the many reform initiatives that have surfaced since the 
publication of A Nation at Risk in 1983. A recent report published by the American 
Federation of Teachers (Gandolf, 1995) stated that 49 states are involved in standards-based 
reform and 3 1 have statewide testing programs linked to these standards, or plan to develop 
these tests. The voluntary national tests proposed by the Clinton administration are an 
example of a federal attempt to set standards for what students should know and do. 



Types of Standards 

Given the plethora of standards in education today, we need to be clear about which type of 
standards we are referring to when discussing standards-based reforms. The project 
described in this paper focused on implementing standards for student learning. Although 
other types of standards such as teaching or schooling may be components of some 
standards-based reforms, they are not the focus of this paper. 

Standards for student learning are generally broken into three types: content standards, 
performance standards, and opportunity to learn standards. Ravitch (1995) distinguished 
each type as follows: 

• content (or curriculum) standards define what teachers are supposed to teach 
and students are expected to learn. 

• performance standards define degrees of mastery or levels of attainment. 

• opportunity to learn standards define the availability of programs, staff, and 
other resources that schools, districts, and states provide so that students are able 
to meet challenging content and performance standards. 
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In addition to a plurality in types of standards, there are also differing views of how to define 
standards-based reform and standards-based education. Attempting to make sense of the 
variety of reform initiatives present in education today, Conley (1997) has identified twelve 
dimensions of restructuring. Reforms based on standards are directed at what Conley has 
labeled the central variables of restructuring: learning standards, curriculum, instruction, and 
assessment. In contrast to other reforms such as site-based management or the use of 
technology, standards-based reforms directly target the areas that are the ultimate target of 
any reform. Conley also notes that change in the central variables is most difficult to achieve 
and “when developing ‘restructuring’ strategies, most educators appear to prefer to look first 
at change in almost anything other than these variables” (p. 114). 

In 1995, the National Academy of Education Panel on Standards-Based Education Reform 
included the following components in their definition of standards-based reform: 

Setting standards of performance in academic subject areas as a means of improving 
the substance of school curricula and increasing the motivation and effort of 
students, teachers, and school systems and thereby improving student achievement. 
The reform assumes high standards for all students and has two components: 
challenging standards that set out what students should know and be able to do, and 
an accompanying agenda for educational equity (p. 70). 

A different view is offered by Massell, D., Kirst, M., and Hoppe, M. (1997) based on their 
study of standards-based reforms in nine states. They report that in addition to setting 
standards and developing assessments, many states link other reform initiatives to standards. 
Examples of these additional reforms are as follows: site-based decision making, open 
enrollment, teacher certification, and accountability systems. To encompass the broad focus 
of these reforms thty use the term “standards-based systemic reform” and define it as 
follows: 

• Establishing challenging academic standard for what all students should know 
and be able to do; 

• Aligning policies - such as testing, teacher certification, and professional 
development-and accountability programs to standards; and 

• Restructuring the governance system to delegate overtly to schools and districts 
the responsibility for developing specific instructional approaches that meet the 
broadly worded standards for which the state holds them accountable. . . the 
overarching objectives of this policy approach are to foster student mastery of 
more rigorous, challenging academic content and to increase the emphasis on its 
application” (pp. 1-2). 
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In addition to de finin g standards-based reforms, researchers have also attempted to define 
what standards-based education would look like in practice. According to Conley (1997), 
standards-based education generally differs from traditional education because instead of 
receiving credit for the amount of time spent receiving instruction, a student must achieve 
the performance level specified in the standard. In other words, student are promoted to the 
next grade based on achievement of performance standards rather than on the number of 
courses taken. He also emphasizes that in a standards-based system, the learning standards, 
the method of assessing student performance, and the required level of performance needed 
for passing are generally set externally rather than by individual teachers as they are in a 
traditional system. 

Harris and Carr (1996) defined a set of criteria for schools to use in assessing their progress 
towards a standards-based curriculum. These criteria cover five areas: standards, 
development and selection of activities and resources, products and performances, criteria, 
and scoring guides. They describe the most accomplished level of implementation as 
follows: 

• Standards form the basis of overall consideration of the curriculum, instruction, 
and assessment within and beyond the classroom. 

• Activities and resources are developed as part of a long-range plan to ensure 
student attainment of standards over time. 

• Products and performances are planed to provide a cumulative record of 
evidence of students’ attainment of standards over time. 

• Criteria address separate standards and define overall performance across 
standards. 

• Scoring guides are anchored in exemplars of student work with benchmarks at 
each level (p. 55). 

They also note that when teachers are using a standards-based curriculum their 
communication with students, parents, and the community is based on student progress 
towards the standards. 

As noted by Massell, Kirst, and Hoppe (1997), another reform strategy often used in 
conjunction with standards is that of accountability. Although it is possible to have 
standards in place without a corresponding system for holding schools or students 
accountable for meeting them, it is not possible to have accountability without standards. 
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Newmann, King, and Rigdon (1997) define a complete school accountability system as 
having, at minimum, these four components: 

1. Information about the organization’s performance (e.g., test scores). 

2. Standards for judging the quality or degree of success of organizational 
performance (e.g., a mean achievement score higher than other schools with 
comparable demographic characteristics). 

3. Significant consequences to the organization (i.e., rewards and sanctions such as 
bonuses to teachers in the school) for its success or failure in meeting specified 
standards. 

4. An agent or constituency that receives information on organizational 
performance, judges the extent to which standards have been met, and 
distributes rewards and sanction (e.g., the state department of instruction) 

(p. 43). 

McLaughlin, Shepard, and O’Day (1995) make an important distinction between using 
performance standards for accountability versus using them for certification. The latter 
refers to situations where results are reported for individual students and thus used to certify 
their achievement. In some instances, the assessment results are only reported on 
aggregated levels and are thus used to hold parts of the education system accountable. 
Darling-Hammond and Falk (1997) make the important point that "ultimately, accountability 
is not only about measuring student learning but also about actually improving it" (p. 193). 

Why should standards and accountability systems improve student learning? How should 
setting standards lead to increases in student achievement? The next section summarizes 
various perspectives on the theory of change in standards-based reform and then discusses 
what is known about the impact of standards-based reform on teaching and learning. 

Implementing Standards-Based Reforms 

Research and writing on standards-based reform tends to focus on the content of standards 
or who should set standards. Little is known about how to move from setting standards to 
actually using them to change instruction and improve student achievement. The 
assumption behind standards-based policies is that if an external agent defines standards and 
mandates assessments for these standards, then the whole education system will change in 
whatever ways are needed to help students achieve the standards. As stated by McLaughlin, 
Shepard, and O’Day (1995) “standards-based reform developed out of the common sense 
notion that student effort and level of achievement are direcdy affected by the expectations 
that have been set. It is assumed that by holding schools and students accountable for 
meeting performance standards, they will be motivated to undertake major changes” (p. xv). 

Two researchers have begun to explore the details of implementation and based on previous 
research with earlier reforms, several theorists have described their own ideas of how to 
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successfully implement standards-based reform. These perspectives will be reviewed in this 
section. 

Based on a long-term study of standards-based reform in nine states, Massell (1997) has 
noted that states use a range of strategies to bridge the gap between state standards and 
classroom practice via curriculum. She described this range as “from efforts to leverage 
change in the commercial publishing industry, sponsorship of curriculum development, or 
sponsorship of curriculum development as part of professional development, evaluation and 
dissemination about existing curricula ...” (p. 4). 

In another report based on the same study of standards-based reform in nine states, Massell, 
Kirst, and Hoppe (1997) observed the following issues related to implementation: 

• Local educators find standards beneficial because they provide focus for instruction 
but they are stated too broadly. 

• Teachers develop assessments or standards in some states, which provides 
professional development. 

• The implicit or explicit assumption in policies that “district or school staff would 
have the capacity, resources, time and expertise to flesh them out [standards] into a 
local curriculum” (p. 23) places a huge burden on local educators. 

• Teachers need direction and support from the district. “Policymakers in recent years 
have ignored the role of the district administrators and local boards, frequently 
conceiving of them as impediments to be bypassed rather than partners in the 
change effort. Yet these administrators are often pivotal conduits for reform 
interpreting its substantive and providing— or not providing, as the case may be— 
both organizational structures and resources that effect whether and how they are 
translated into school and classroom practices” (p. 63). 



In a recent study, Newmann, King, and Rigdon (1997) conclude that external accountability 
mechanisms alone are not sufficient to improve school performance. Based on data 
gathered from twenty-four restructuring schools, they propose that three factors influence 
the likelihood that external standards will lead to increases in student achievement. 

The first factor is organizational capacity, which Newmann et al. (1997) define as, “the 
degree to which the human, technical, and social resources of an organization are organized 
into an effective collective enterprise” (p. 47). In a school with a high organizational capacity, 
all resources are directed toward a clear, shared purpose for student learning. Teacher 
knowledge, leadership and a school’s technical and financial resources are effectively 
integrated to accomplish the school’s vision for student achievement. 
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The second factor is internal accountability. Schools with internal accountability do not rely 
on an external agent to prescribe standards and provide incentives and sanctions. Instead, 
staff within the school exerts peer pressure to meet internal standards for performance and 
the school has a process for collecting information to monitor progress on these standards. 

The third factor is the difficulty created by controversies over how to implement standards. 
These include conflicting views on what the content of standards should be and who should 
establish the standards. To overcome the difficulties created by these controversies, 
Newmann et al. (1997) propose that policies designed to increase school’s external 
accountability should include efforts to increase the organizational capacity of schools and 
support schools to develop internal accountability. 

Their examination of these factors in 24 schools led them to conclude the following: 

• Even when a strong external accountability system is present, schools that do not 
have organizational capacity to address these challenges will not improve their 
performance. 

• It is possible for a school to develop strong internal accountability even without 
external mandates. 

• Highly prescriptive standards linked to high-states consequences through a mandate 
by external authorities “can deny school staff both the ‘ownership’ or commitment 
and the authority it needs to work collaboratively to achieve a clear purpose for 
student learning” (p. 62). 

A different perspective on standards implementation comes from Darling-Hammond (1994). 
She believes that implementation should begin by building the capacity of teachers and 
schools, rather than starting with content and performance standards. With their increased 
level of knowledge, teachers and schools will be better prepared to implement this major 
change. The role of state and federal governments in this model would be “to create the 
policies that ensure that teachers and schools have the knowledge, the material resources, 
and the support for school-based equity that will allow them to create good curriculum for 
the students they serve” (p. 482). 

There are two major, somewhat related reasons for Darling-Hammond ’s (1994) emphasis on 
building capacity as a first step in standards implementation. First, because she believes 
students have varying routes to learning and content and performance standards that are 
determined “on-top” can not accommodate this diversity. Instead, teachers need the 
knowledge of how to adapt instruction to help every student achieve their potential. If they 
are just handed a set of standards they are unlikely to develop the skills to make these crucial 
instructional modifications. “To be effective, teachers must meet students where they are, 
not where an idealized curriculum imagines they should be” (p. 491). Her second reason 
comes from research on educational change generally. She refers to the research of Fullan 
(1994) and stresses the importance of teacher understanding of a reform to the potential 
success of the reform. By having teachers develop standards and assessments locally, they 
are provided with the opportunity to work through the standards. This process, she states, 
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“is in itself educative and supportive of change as it stimulates inquiry into local practices in 
relation to new standards that have engaged local ownership” (p. 488). 

Another component of standards-based education that Darling-Hammond (1994) 
emphasizes is the importance of providing teachers with information they can use to adapt 
their instruction to individual student needs. If the performance standards are used only to 
sort and select students into more and less learning opportunities, she believes there is little 
chance that standards will improve student achievement as intended. 



Background on Minnesota’s Standards Movement 

The current policy environment in Minnesota provides a good context for further study on 
the implementation of standards-based reform. The basic standards and high standards are 
externally imposed by the state department of education (hereafter MDE). Currently, the 
state rule requires students to achieve passing scores on state-developed basic skill tests in 
reading, mathematics, and writing in order to graduate from high school. The assessment 
packages for the high standards, however, provide school districts with much more 
flexibility. Because state law forbids a mandated state curriculum, each district can modify 
the assessment packages to fit their local context and the state does not specify how to 
instruct the students. In order to prepare students to meet the state standards, it appears 
that districts must already have, or must develop, the org anisa tional capacity defined by 
Newmann, King, and Rigdon (1997). As described earlier, one component of organisational 
capacity is the skills and knowledge teachers will need to effectively practice in the changed 
environment. The significant bottom-up component of Minnesota’s reform approach also 
provides an opportunity to study effective means for building teacher capacity, the primary 
emphasis in the writing of Darling-Hammond. 

Minnesota schools have been hearing about standards-based reform for seven years, 
however, it has yet to develop into a full accountability system as elaborated by Newmann et 
al. (1997). In May 1991 the Minnesota State Board of Education gave preliminary approval 
to a new system based on graduation standards and MDE began developing the standards 
and assessments for this new system. Instead of determining graduation based on the 
completion of the number of courses and credits, student would graduate only by 
demonstrating achievement of the student learning standards. Two years later, the Board 
selected thirteen pilot sites to continue the development work in conjunction with MDE. 
Each year the number of pilot sites has increased, and based on data gathered from the pilot 
sites and public reaction, the standards and assessments have continued to evolve. 

The Minnesota K-12 Graduation Standards are made up of two separate pieces: basic 
standards in reading, mathematics, and writing composition; and high standards in ten 
learning areas such as inquiry and math applications. The high standards are interdisciplinary 
and not tied to specific subject areas or courses. Therefore, each district must decide which 
standards will be covered in which courses. This may result in variation across districts in 
how students prepare to achieve the standards. 
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The basic standards reflect minimum competencies in essential skills for graduates. They are 
assessed by state developed tests which are first administered to students in the eighth grade. 
The reading and math tests are multiple choice form and writing composition is assessed 
through a writing sample. 

In contrast, the high standards define what it takes for students to know or do something 
very well. They are oriented toward excellence and proficiency rather than minimum 
competence. The content of the high standards was frequently revised during the seven-year 
development period but currently the standards are going through the rulemaking process 
and approval is expected this spring. The high standards are assessed by “performance 
packages” developed by MDE in conjunction with pilot sites. The packages are made up of 
performance tasks that require students to demonstrate their knowledge and skills in a real- 
world context. The high standards allow more flexibility at the local level because teachers 
can change the topic of a package and students can select from multiple standards within 
each area. 

Minnesota law forbids a state curriculum, therefore the high standards have been written to 
provide latitude for local interpretation and curriculum. The standards focus on what a 
student should be able to know and do but do not include much content because this is to 
be determined locally. Assessment packages for the high standards were developed by pilot 
sites and curriculum is expected to be developed locally. This system exemplifies what is 
known as a “bottom-up and top-down approach.” The state developed the standards and 
assessments, in collaboration with statewide educators and community members, but it is up 
to local districts to determine how to prepare students for the assessments. For example, 
one district may choose to teach students inquiry skills through a semester long science 
project and another district may choose to teach the same skills through a social studies 
course. 

This system also requires development of teacher capacity because teachers are required to 
do things they have not done before. For example, the high standards require teachers to do 
the following: teach and assess different skills, which are embedded in the standards; develop 
instruction aligned to the standards; and modify instruction as needed to prepare each 
student for success on the assessments. 

The 1996 freshman class will be the first to graduate under the new system of graduation 
standards; they are required to achieve basic requirements in reading and math. The writing 
composition requirement becomes effective for the 1997 freshman class. Due to its lengthy 
and controversial development process, the high standards component does not become 
effective until the freshman class of 1998, if the rule is approved at that time. Although 
school districts have not yet had to withhold diplomas from students not passing the basic 
skills tests by the end of the twelfth grade, district level scores are made public by the 
Department and many districts feel pressure from their constituencies to release results by 
school. 

In 1997 a statewide testing and reporting system was established to assess student 
achievement of the standards. In addition to the state basic skills tests, students in the 3 rd 
and 5 th grade are now required to take the Minnesota Comprehensive Assessments to 
measure their progress towards the high standards. The legislature also required the 
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Commissioner to recommend legislation on performance funding options that would tie 
state education funding to school performance. The current education funding law expires 
in 1999. Also in the 1997 session the legislature authorized funding for an Office of 
Educational Accountability, an office to be housed in an institution of higher education to 
maintain its independence from the Department. 

Looking back at the accountability system components delineated by Newmann, King, and 
Rigdon (1997), we can see that Minnesota currently lacks the minimum contents of an 
accountability system. Testing and publication of scores on the basic skills tests and 
comprehensive assessments is in place but only the basic skills tests have established 
standards for judging performance. The current school year is considered a baseline year for 
establishing standards on the comprehensive assessments. The department continues to 
work with schools to develop skills in assessing the high standards and reporting systems to 
track student results. The procedure for establishing standards for student performance on 
these assessments is unclear and may be done locally. Finally, only in the last legislative 
session did Minnesota establish an office to receive and judge results and a study of 
performance funding options is underway. 



Implementing Minnesota’s High Standards for Student Learning 

The focus of this paper is a project designed to develop and pilot-test a process for 
implementing Minnesota’s high standards for student learning. The project, Implementing 
Academic Standards (hereafter called IAS), was funded by a 2 million dollar grant MDE. It 
was a collaborative endeavor among seven diverse Minnesota school districts and a for- 
profit software developer, which will be called NEI in this paper. The project covered an 
18-month period from March 1996-August 1997. 

As described above, MDE had already developed the high standards for student learning and 
corresponding performance assessment packages for most of these standards. IAS believed 
the next step in implementing the high standards was to have teachers develop learning 
activities linked to the standards. Because the high standards were relatively new and are 
unique to Minnesota, there were no curriculum packages or textbooks that districts could 
purchase to guide instruction. At this point in Minnesota's standards implementation 
process, no one had yet tried to develop classroom activities linked to the standards. IAS 
strongly believed that unless a process was developed to help teachers incorporate the 
standards into their daily classroom practice, then the standards would not be effective in 
changing instruction and improving student achievement. 

By choosing to create a process for teachers to develop standards-based instructional 
activities, the project's approach to standards implementation resembles the strategy 
described by Massel, Klrst, and Hoppe (1997) as changing instruction through professional 
development 1 . The approach also addresses some of the components of Newmann, King, 
and Rigdon ’s (1997) organizational capacity by trying to increase teachers' skills and 



1 Minnesota is one of the states included in their study. 



knowledge of how to use standards to guide instruction, thus enabling them to make 
decisions about curriculum, instruction, and assessment. 

It is more difficult to describe this project in relation to the implementation model proposed 
by Darling-Hammond (1997). Although teachers in several districts worked with MDE to 
develop the high standards, the state is mandating that districts follow a common set of 
standards rather than allowing each district to develop standards within their local context. 
Unlike other top-down systems, however, the high standards and corresponding assessments 
were intentionally designed to provide a lot of flexibility for local adaptation. As yet, the 
state is not mandating that districts use any particular assessment packages 2 . Instead, each 
district must establish a plan for integrating the standards into their curriculum and must 
assess students on the high standards using either state-developed assessment packages or 
assessments they develop locally. This situation requires districts to do what Darling- 
Hammond (1994) describes as “working through .... [which is] in itself educative and 
supportive of change as it stimulates inquiry into local practices in relation to the new 
standards that have engaged local ownership. It also allows the construction of curriculum 
that is connected to students’ experiences, cultures, and communities by building on prior 
knowledge and understandings in the course of reaching for new concepts and skills” (p. 
488). By building teacher capacity to develop instruction linked to the standards, the project 
would, as Darling-Hammond (1993) advocates, “create bridges between the very different 
experiences of individual learners and the common curriculum goals. . . . Teachers must 
diversify their practice so that they can engage each of their students in whatever ways are 
necessary to encourage learning” (p. 754). 

Another goal of the project was to establish a network of teachers who were implementing 
the standards. They felt it was important to encourage collaborative work for several 
reasons. First, they believed it was unrealistic to expect each teacher to develop their own 
set of instructional activities for the standards. Second, they recognized that in a standards- 
based system a teacher may need multiple lessons for students who are not successful the 
first time (as described by Darling-Hammond above) adding further to the burden of writing 
instructional plans. By working together teachers could share ideas for what kinds of 
activities worked for what kind of students and benefit from each other’s experiences. 

Finally, experience with other reforms and previous work on the state standards had shown 
many members of the project leadership the value of working collaboratively. 

The project attempted to stimulate collaboration in several ways. First, the sites established 
a common template for writing and recording learning activities. The purpose of the 
template was to standardize the language used by teachers to describe their instruction and 
to ensure that sufficient information was recorded so that another teacher could use an 
activity developed by someone else. Second, the project provided each participant with e- 
mail access so they could collaborate electronically and did not have to meet in person. 

Third, the learning activity template was incorporated into a software package so activities 
developed by teachers could be stored in an electronic database, or library. 



2 In current policy, the assessment packages can be developed locally or sites can use or modify packages developed by the 
State. The newer Minnesota Comprehensive Examinations, which were given for the first time at the 3 rd and 5 th grades and 
will ultimately also be given at the 11th grade, are mandated assessments of the high standards but current policy does not 
state that student diplomas will be withheld from students not reaching a certain score on the test. 



o 

ERIC 



12 



10 



The intention was to have both a library at each project site that would include activities 
developed by teachers in that district, and a much larger library of activities located on a 
website maintained by the software developer. This website, known as UnitHouse, would 
contain activities developed by teachers throughout the state. It was hoped that the 
electronic libraries would encourage teachers to deprivatize their practice and collaborate 
with each other by 1) publishing activities they had developed in the libraries, and 2) using 
the libraries to locate activities relevant to their practice and downloading to modify, or use 
as is, in their own classrooms. 



Research Questions and Methods 

We were hired by IAS to document their development of a process for integrating 
Minnesota’s high standards for student learning into classroom instruction, assess the 
effectiveness of this process, and identify effective practices for building teacher capacity to 
implement standards. Our study attempted to answer the following questions: 

What process did IAS develop for integrating Minnesota’s high standards into 
classroom instruction? What did IAS learn about effective practice for integrating 
high standards into instruction? 

To what extent does the process IAS developed increase teacher capacity to 
implement Minnesota’s high standards for student learning? 

To what extent did IAS develop a network of teachers who are working to 
implement Minnesota’s high standards for student learning? 

To what extent did IAS increase the level of standards-based practice among 
teachers who participated in the project? 



Data Collection Methods 



We used three data collection methods in our study: observations, interviews, and a survey. 
Each is described below. 

Observations 

In order to document the process IAS developed for integrating Minnesota’s high standards 
into classroom instruction and identify effective practices, we observed meetings of the 
project leadership team and teacher training sessions. The leadership team (hereafter LT) 
included a representative from each of the eight participating districts 3 and a representative 
from NEI, the software development company. A representative of one district also served 
as the project coordinator. Other district and NEI staff attended the meetings occasionally 
to provide input as needed. 



3 One large urban district bad two representatives. Given the size of tbe district and its emphasis on site-based 
management, each school essentially represented a distinct pilot site for purposes of this project. 
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We attended all but one of the 20 LT meetings that were held throughout the project’s 
implementation. We took notes during each meeting and these notes, along with official 
minutes of each meeting, were used in this study. 

We also attended teacher-training sessions at five sites during the summer of 1996, and 
attended follow-up training that took place during the summer of 1997. During each 
session, we took notes and had informal conversations with the participants and trainers. 

Interviews 

In addition to observing project activities, we interviewed members of the LT and a sample 
of teachers. In September 1996 and September-October 1997, we interviewed each LT 
member. The interviews consisted of a series of open-ended questions. In the first round of 
interviews, the questions focused on their experiences with the initial training sessions for 
teachers. The second round of interviews addressed each stage of the project’s 
implementation process. In addition, we contacted other people in some of the sites to 
gather more complete information about the implementation process at those sites. 

After the project ended, we interviewed 10% of the teachers who had participated. The 
purpose of these interviews was to supplement the written survey of teachers (described 
below) by gathering in-depth information about how they developed instructional activities 
both before and during the project. We established the interview sample by asking the 
coordinator at each site to identify the teachers who had been most involved in the project. 
Because the level of participation by teachers varied considerably within and across sites, we 
intentionally decided to interview those who were most involved in order to have the best 
chance of getting information about how the project may have changed the way they 
develop instructional activities. Given the difficulty many sites experienced with making the 
software available to teachers during the project, we did not require that the teachers in the 
sample had had extensive experience using the software to write activities. 



Written Survey 

The written survey we used in this study is based in part on the professional community 
index described in Louis, Marks, and Kruse (1996). We selected items that corresponded 
with the shared commitment and collaborative activity component of Newmann, King, and 
Rigdon’s (1997) organizational capacity and modified them to fit the context of this study. 
Because professional community is defined as a characteristic of the school as a whole, we 
had to adapt the definition a bit for our study. Ultimately, the hope of IAS is that all 
teachers in a school, district, and the state would be working to implement standards and 
working together in various combinations across schools, districts, and even states. 
However, in this initial exploratory study the community was defined as teachers that 
worked together to develop learning activities and when they were developing l earnin g 
activities. We also excluded some items because they described behaviors that could not 
have occurred during the project such as “ except for monitoring student teachers or 
substitute teachers, how often have you visited another teacher’s classroom to observe and 
discuss their teaching since the beginning of the current school year?” (p. 789, Louis et al., 
1996). We also couldn’t ask about faculty meetings or lounge conversations because these 
items would involve teachers outside the project. 



In addition to drawing items from the professional community index by Louis, Marks, and 
Kruse (1996), we also wrote new items to measure other study variables such as standards- 
based practices because, in a search of the literature, we did not find any instruments were 
sufficient. We used the criteria for a standards-based curriculum developed by Harris and 
Carr (1996) to develop the items on standards-based knowledge, beliefs, and practices. 

Members of the LT reviewed the first draft of the survey and made suggestions for content 
changes and item wording. After making the suggested revisions, we pilot -tested the next 
survey draft with teachers in almost all of the sites. They provided further useful suggestions 
about wording of items and response options. 

We sent the final version of the survey to the coordinators at each site and they distributed 
them to the teachers whom had participated. The surveys included a stamped return 
envelope addressed to us to assure confidentiality. The first mailing generated a response 
rate of 27% so we sent out additional surveys to the coordinators and extended the return 
deadline. Coordinators were provided with a list of people who had not completed the 
survey and we asked them to distribute the survey to these people. 

Data Analysis 

We collected a large amount of qualitative information during the project and we used the 
following techniques condense it: 

• We reviewed responses to each interview question and clustered similar responses. 

• We reviewed our field notes and the official meeting minutes to identify major 
events and themes. 

Qualitative data collection and analysis procedures are vulnerable to researcher bias because, 
despite our efforts to bracket our biases, we may unintentionally have overlooked or 
minimized the significance of certain events or comments. We used several strategies to 
minimize the potential bias. First, we conducted interviews and observed meetings as a team 
whenever possible to provide us with two perspectives. Second, we asked two colleagues 
not involved in the study to review this report. Finally, the project coordinator and NEI 
staff members reviewed the report in order to identify factual errors or areas where 
alternative interpretations exist. 4 

The survey responses were entered into a database and analyzed using SPSS software. 



4 The other members of the LT were also given the opportunity to review the report but none of them participated. 
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Results 



This section summarizes results of the study and is organized by the research questions. 
First, however, we’ll review the response rates for the survey and the interviews. 



Response Rates 

One site dropped out of the project in the last quarter and chose not to participate in the 
survey or teacher interviews. The 26 teachers from this site are not included in further 
totals. Interviews were completed with each member of the LT and in some sites we 
contacted additional administrative staff to gather more complete information. 

We attempted interviews with 26 teachers. We completed 20 interviews (77%); the 
remaining teachers either chose not to participate or did not return phone calls (at least four 
messages were left). The percent of teachers completing the interview was fairly constant 
across sites with the exception of site G where only two of the four teachers agreed to an 
interview (see Table 1). 



Table 1: Interview Completion bv Site 



Site 


Number of teachers in 
project 


Number completed/Number 
attempted 


A 


36 


3/4 


B 


30 


2/3 


C 


42 


4/4 


D 


19 


2/2 


E 


14 


2/2 


F 


48 


6/7* 


G 


42 


2/4 


Total 


231 


20/26 



a The number of interviews attempted for this site was based on the total number of participants (n=67) 
including the 19 that were not included in the written survey because they dropped out of the project after the 
first training series. 



We mailed surveys to 23 1 teachers. This total does not include the 26 teachers from the site 
that dropped out of the project; also excluded were 19 teachers at one site who dropped out 
of the project after the first training series. Ninety-one of the teachers completed the survey 
for a response rate of 39%. As shown in the table below, two sites had 10% or fewer of 
their participating teachers complete the survey (one of these also had the lowest response to 
the interview). Given these, low response rates, we can not know whether the results in this 
report are representative of the teachers at these two sites. 
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Over half of the respondents (56%) teach in an elementary setting. The number of years 
respondents have been teaching varies from 2 years to 30 years; 54% had been teaching ten 
years or less. 

Currently, we are attempting to contact a random sample of non-respondents by telephone 
to determine if they are different on key variables than those who returned the survey 
through the mail. This effort will provide more information about the risk in generalizing 
from the respondents to the written survey to all IAS participants. 



Table 2: Survey Completion by Site 



Site 


Number of teachers in 
project 


Number /Percent of Surveys 
Completed 


A 


36 


0 0% 


B 


30 


14 47% 


C 


42 


23 55% 


D 


19 


15 79% 


E 


14 


6 43% 


F 


48 


29 60% 


G 


42 


4 10% 


Total 


231 


91 39% 



A Process for Developing Standards-Based Instruction 

One objective of IAS was to develop a process teachers could use to integrate Minnesota’s 
high standards into their classroom instruction. As will be described below, the LT needed 
to make significant changes in their plans during this stage of the project. Although these 
changes meant that the project got behind schedule and took up much more time on the 
part of everyone involved, a majority of the participants would agree that this stage of the 
project was the most productive. 

Initially, the LT assumed that after providing teachers with training on how to use the 
software (into which the IAS template for writing learning activities had been incorporated) 
the teachers would be able to work on their own, or in teams, to develop instructional 
activities. Thus, the primary focus of the training was on how to use computers and how to 
use the software. Some of the sites also included an overview of the standards in their 
training, but all sites seemed to assume that once teachers became familiar with the software 
then they would be able to develop standards-linked instructional activities. Many of the sites 
had difficulties maintaining' their computer networks, which contained the software, during 
the training due to an insufficient level of technical support within their districts. These 
disruptions to the software caused many teachers to become frustrated in their attempts to 
learn how to use the software. 
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As teachers began to develop learning activities during the first summer of the project, they 
encountered substantial obstacles, in addition to problems accessing the software. Some of 
the issues they experienced were as follows: areas of the standards were ambiguous or 
contradictory', the distinction between an instructional activity and an performance 
assessment was unclear, they were unsure how the instructional activities they were 
developing should fit in with existing district curricula and scope and sequence for 
instruction. To work with the project in clarifying these issues, the software developer hired 
a specialist in curriculum and assessment who had worked for the state department of 
education during an earlier phase of the standards initiative. 

One change that resulted from this woik was to establish a clear distinction between a 
learning activity and an assessment. The specialist and the project leadership agreed that in a 
performance task, evaluation is the major purpose of the assignment. In contrast, the major 
purpose of a learning activity is instruction and practice. To provide students with 
information and time to practice their skills in a non-judgmental manner, a checklist of 
performance criteria in a learning activity should only be used to provide feedback to the 
student and not for grading purposes. 

The specialist also recognized that teachers were writing learning activities that looked like 
miniature versions of the assessment package for the intended standard. She worked with 
the teachers to identify what students needed to learn and how to break this down into 
multiple activities. 

The most significant change in the project occurred in response to teachers’ questions about 
how the activities they were developing fit with district curricula and other activities the 
teachers were already using. The project’s model for implementing the standards, as it was 
originally conceived, started with the teachers. It was believed that teachers needed training 
to shift their thinking about instruction to a results rather than activities orientation. The 
assumption was that as teachers wrote learning activities linked to specific standards this 
shift would occur. 

As described earlier, individual differences and district autonomy have always existed in 
Minnesota’s education system and the standards are designed to preserve this autonomy by 
allowing for a great deal of flexibility based on local contexts. In a traditional education 
system, local curriculum design is heavily influenced by textbook selection; under the new 
model, desired results drive curriculum design. Based on their early experiences in having 
teachers write standards-based activities as special units outside the district curriculum, the 
LT recognized that teachers could not develop learning activities in isolation. Teachers 
needed to know which results, i.e., which parts of the standards, were their responsibility and 
which would be addressed elsewhere in the curriculum. Thus, district level leadership was 
needed up front to make decisions about how to interpret and place the standards in the 
overall district curriculum. 

To assist districts in placing the standards in their overall curriculum, the curriculum and 
assessment specialist developed a process called “sketching the standards.” The process was 
pilot tested with representatives from the project sites during four workshops held from 
February to April 1997. The process was later revised based on feedback during the pilot 
and subsequent work with some sites after the workshop. The complete process is 
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described in a publication available from the software developer 5 . The major steps are as 
follows: 

1. Determine the course in which the standard’s final assessment and rating will 
take place. 

2. Identify and sketch the “pattern” of the standard. A pattern displays the standard 
graphically', this assists staff and teachers in translating the standard into a format 
that is useful for curriculum development. There are three possible patterns in a 
standard. The first, a big project, is used for standards that describe a multi-step, 
sequential process. The second, a content guide, is used for standards that 
consist of hsts of topics or ideas that should be taught and assessed. The last, a 
template, is used for a standard that requires a student to carry out a sequence of 
steps repeatedly, using a different topic each time. 

3. Create a sketch for the course in which the standard will be taught and assessed, 
the course into which the standard was placed in the first step. The sketch 
includes the following: 1) the pattem(s) and requirements of the standard; 2) 
assessments to rate students on the standard; 3) content from district curriculum 
guidelines; and 4) instructional decisions made by individual teachers. By 
sketching out the requirements of the standard, teachers can see where there are 
gaps, and where they need to add content, instruction, or assessment from the 
district curriculum or their own preferences. A standard may be taught in a 
single unit in a course or in multiple units. 



Developers begin the sketching process with the course in which the standard will be 
assessed and assume that all the content and skills required for a given standard can be 
taught and assessed in this course. If they discover that this is not feasible, they identify 
course(s) earlier in the sequence that could include the necessary content and create 
additional sketches. 

As workshop participants worked to create their graphics, the level of complexity in the 
standards became more apparent. Participants made the following comments after 
participating in the workshops: 

It's more than placing the standards; it's tearing them apart. 

We've never had any way to break down the standards. What does it mean day to day? 

The ultimate goal of the sketching is to have a scope and sequence of content . . . 

people are really exited about this. 



McGinnis, I. (1997). Sketching the Standards: A Strategy for Planning Standards-Referenced CurriculurrL (2nd ed.). 
Minneapolis, MN; Norris Education Innovations, Inc. 



After the workshop, members of the leadership from three sites met several times to 
continue drawing patterns for the standards. By the end of the project they had completed a 
pattern for every standard at each of the four levels: primary, intermediate, middle, and 
graduation. 

Not all sites participated in developing the sketching process and the sites that assisted in the 
development varied in their use of the process after the training. The differences are as 
follows: 

Four sites participated fully and their administrators have continued to sketch 
standards after the training was completed. Teachers at one site later used the 
patterns to develop a scope and sequence for the following areas: people and 
cultures, scientific applications, and math applications for grade K-6; and resource 
management for grades K-12 6 . 

One site said the process seemed like it would fit well with their needs, but they 
had not done any sketching yet. 

One site began the process but dropped out because the language and framework 
being used were inconsistent with their site’s commitment to curriculum and 
instruction that is multicultural. 

One site already had its own process for doing this and did not participate after the first 
session. One member described the decision not to participate further as follows, “What I 
saw resulting is the standards being used to drive the total district curriculum. Rather than 
the process that we’re using - I have a district curriculum, I want to know where the 
standards fit. So for me it’s a matter of fitting, for other people it’s a matter of generating 
curriculum and using the standards as the basis for that.” 

The process may, at least in part, address a common barrier to real reform as described by 
Darling-Hammond (1993): 

A massive geological dig would be required to unearth the tangled influences that 
created the many layers of policy that people in schools must now contend with. 
These influences make the serious implementation of new policies difficult, even 
impossible, without excavation and reform of what has gone before (p. 756). 

The sketching process asks teachers to put everything on the table that they need to consider 
in developing instruction: the state standards; district and national standards, in some cases; 
district curricula; the teachers’ own preferences for instruction; student needs; and 
available resources. In developing sketches of the standards, teachers are able to integrate 
these sometimes disparate pieces and identify gaps, areas of overlap, and instructional 
priorities. In some cases, creating sketches of the standards may be the first time a teacher 
has considered all of these instructional influences simultaneously. Sketching also forces 
communication across grade levels and classrooms within grade levels because teachers 



6 The standards only go through grade 9 but the district scope and sequence goes through grade 12 to include knowledge 
and skills that the district thinks is necessary in this area. 
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recognize the need to coordinate their instruction in order to adequately prepare students to 
achieve the high standards. 



Increase Teacher Capacity to Implement Standards 

After the project ended, teachers who completed the survey were asked about changes in 
their knowledge and beliefs related to standards-based education. These areas were 
measured by six items on the survey (see Table 3). The greatest change occurred on the item 
“I have sufficient understanding of how to develop learning activities linked to the 
standards.” The average rating on this item increased by 1.2 rating points. The next largest 
change was on a similar item, “I feel confident about my ability to link learning activities to 
the standards.” The average rating on this item increased by 1.1 rating points. 

All but one of the items showed a statistically significant increase as measured by a matched- 
pair’s t-test. The only item not showing a significant change asked teachers about their belief 
that the standards were a fad. The lack of change in this item could be attributed to 
selection bias in the teachers who volunteered for the project. Those who believe the 
standards are a fad would be unlikely to take time to implement the standards. 



Table 3: Change in Knowledge and Beliefs Related to Standards 
Rating was on a scale of 1 to 5 where 1 was strongly disagree and 5 was strongly agree 



Item and (n) 


Mean/Mode 
Rating Before 
Project 


Mean Rating 
After Project 


Mean Change 


I have sufficient understanding of how to 


2.6 


3.8 


1.2* 


develop learning activities linked to the 
standards. (n=88) 


3.0 


4.0 




I have sufficient understanding of how the 


2.7 


3.7 


1.0* 


standards fit with our district’s K-12 
curriculum. (n=89) 


3.0 


4.0 




I think the standards are just another 
passing educational fad. (item scale 
reversed, n=85) 


3.5 


3.3? 


-.2 


I am confident about my ability to link 


2.9 


4.1 


1.1* 


learning activities to the standards. (n=89) 


3.0 


4.0 


I believe that using standards can make a 


3.2 


3.8 


.6* 


positive difference in student achievement. 
( n=89 ) 


3.0 


4.0 


I believe that using standards can make a 


3.2 


3.9 


.7* 


positive difference in my teaching. (n=89) 


3.0 


4.0 
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Another indicator of whether the project increased teacher capacity to develop standards- 
based instruction is the number of learning activities teachers completed during the project. 
The number of completed activities reported by teachers on the survey ranged from none to 
eighteen. Most often (28% of the respondents), a teacher reported completing three 
activities. Only 20% wrote more than four activities. If we expect teachers to change how 
they instruct and assess as a result of this project, it seems that teachers would have to have 
written more activities and had the opportunity to use them in the classroom before the 
project ended. 

Based on these results, it appears that teacher knowledge and beliefs related to the standards 
changed in a positive direction during the study. Because there is no comparison group and 
many sites had other standards-related initiatives occurring during the IAS project, the 
results can not be used to conclude that the project alone or in combination produced these 
changes. 



Building a Network of Teachers Implementing Standards 

An important aspect of this project was to establish a network of teachers who were working 
together to develop and use standards-based learning activities. As described earlier, the 
project undertook several activities to facilitate collaboration among participating teachers: 
creating a common template for writing and recording learning activities, providing an 
electronic database of these activities, and giving teachers Internet access and e-mail. In this 
section, we look at the extent to which a network developed during the project. 

The survey asked teachers several questions about their level of collaboration with colleagues 
on the project. As shown in Table 4, almost half of the teachers reported that they always 
worked alone (19%) or primarily worked alone (28%). Only 8% of the teachers always 
worked with others. 

Of the teachers who reported working with others at least some of the time, half spent less 
than four hours collaborating with colleagues to develop learning activities. The other half 
spent anywhere from five to 200 hours collaborating and the most typical length of time 
reported was nine hours. 

When they collaborated, teachers were most likely to work with other teachers in their 
building who teach at the same grade level (41%) or in the same content area (36%). This 
disparity may be due to a greater number of elementary teachers returning the survey. If 
they collaborated with teachers outside their building, teachers were more likely to 
collaborate with others in the same content area (13%) than the same grade level (6%). The 
primary mode of communicating with colleagues on activities was in-person meetings. No 
one used faxes and only 3% used e-mail 7 . 



7 The option of communica tin g by accessing someone else’s activities in the database of activities was not included in the 
survey because we knew that school’s could not access the electronic library during the project. 
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Table 4: Level of Collaboration with Colleagues 



Whii bstatemet it be st describes your level of 
adlaboration with colleagues when developing learning 
activities f (N=91) 


n 


% 


I always worked alone. 


17 


19% 


I worked primarily alone. 


25 


28% 


I sometimes worked alone and sometimes 
worked with other teachers. 


27 


30% 


I worked primarily with other teachers. 


13 


14% 


I always worked with other teachers. 


7 


8% 


Did not answer 


2 


2% 



To assess whether the common template for writing and recording learning activities and the 
electronic potential for collaborating may have increased the amount teachers worked with 
others, we asked teachers how their level of collaboration with project colleagues compared 
to how they would usually plan classroom instruction. Of those who worked with others at 
least some during the project, 40% said their level of collaboration to write activities was 
very similar to how they would usually plan instruction. Only 27% said it was not similar or 
not very similar. This suggests that the common format may have encouraged this 27% to 
collaborate more than usual. 

The survey also measured collaborative activity by asking teachers about getting input from 
others as they developed activities. As shown in Table 5, teachers were more likely to 
discuss suggestions for teaching techniques or student activities as they collaborated than 
they were to get input from colleagues about a learning activity. The average response to the 
former item was 4.0 on a scale where l=rarely and 5 =always. The average rating for the 
item about learning activities was lower at 3.1. 



Table 5: Frequency of Input from Colleagues 



Measured on a scale from 1-5 where 1 = rarely and 5 ° always 



When you worked wii bcolleagu esi y owri te learning 
activities, how often were the following topics part of your 
conversation ? (N=69) 


Most 

frequent 


Mean 


Input from colleagues about learning activities I 
was writing. 


4.0 


3.1 


Useful suggestions for teaching techniques or 
student activities. 


4.0 


4.0 



Another method the project proposed to encourage collaborative activity among teachers 
was to establish an electronic library of learning activities linked to the standards. Teachers 
could collaborate through the library either by submitting activities they had developed or by 
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downloading an activity developed by someone else. This portion of the project could not 
be assessed, however, for several reasons. First, as mentioned earlier, due to technical 
problems none of the sites could access the electronic library located on a Website created 
by the software developer. Second, because it took longer to develop learning activities than 
originally intended, neither the local library nor the software vendor’s library had much 
content during the project. 

Given that it was not possible to ask teachers about their actual use of the library, we 
included the following items on the survey: 1) How comfortable are you with publishing 
learning activities you have written on your local UnitMaker system?, and 2) How 
comfortable are you with making your learning activities available for broad public 
dissemination through UnitHouse, the electronic curriculum library maintained by NEI? 8 

Half of the teachers said they would be very or somewhat comfortable publishing learning 
activities they had written in their local library. Just over one-third (34%) said they would be 
very uncomfortable doing this. Comfort with making their activities available broadly was 
similar. Fifty-two percent said they would be very or somewhat comfortable making their 
learning activities available for broad public dissemination through UnitHouse, the electronic 
curriculum library maintained by the software vendor. Thirty percent said they would be 
very uncomfortable doing this. 

In attempt to measure Newmann, King, and Rigdon’s (1997) shared commitment 
component of organizational capacity, the survey asked teachers how much working with 
colleagues to develop learning activities helped establish agreement on the topics listed in 
Table 6. The topic for which teachers most often reported that they established agreement 
when working together is “what students should learn in school.” Forty-two percent 
reported that working with colleagues helped a lot to establish agreement in this area. In 
contrast, only 10% of the teachers said that working together helped a lot to establish 
agreement on “how hard students should work.” 

Given the short amount of time that many teachers spent working together, it is difficult to 
know if, given more discussion, more of them would have reported that the process helped 
create a shared commitment. It is encouraging, however, that so many reported that the 
process helped a lot in establishing agreement about what students should learn. This 
suggests that the process used in the project to create learning activities was effective in 
establishing consensus on what the standard says students need to know and be able to do. 



8 This reference is to the library created by the software developer as a Website. 
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Table 6: Extent that Collaborative Work Established Shared Commitment 



How much did working zvith 
colleagues help establish agreement 
amongyour group on the issues 
below? 


Not at All 


Little 


Some 


A Lot 


How hard students should 
work. (n=70) 


11% 


29% 


50% 


10% 


What students should learn in 
school. (n=69) 


0% 


17% 


41% 


42% 


The beliefs and values about 
what the central mission of 
our school should be. (n=69) 


10% 


33% 


42% 


15% 


The goals and priorities for 
our school. (n=69) 


10% 


28% 


39% 


23% 



Increase the Level of Standards-Based Instruction and Assessment 



Perhaps even more important to the success of standards-based reform is whether teachers 
change their classroom practice to include greater use of standards. This outcome was 
measured by several items on the written survey and questions from the interview. The 
results will be discussed in two sub-sections: developing instruction, and the effect on 
instruction and assessment practices. 

Developing Instruction 

The survey asked teachers to describe their agreement with items describing standards-based 
practices as reflective of their practice before the project and currently. A higher number on 
the current rating indicates that the teacher’s practice is more in agreement with the 
statement. Three items related to teacher’s practices in developing instruction. The 
difference for each teacher’s two ratings was calculated to assess the degree of change. Each 
of the three items showed a statistically significant positive change on a matched pairs t-test 
(see Table 7). The item that showed the largest magnitude of positive change was as 
follows: “I use the standards as the basis for planning learning activities”. 
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Table 7: Change in Use of Standards for Developine Instruction 



Rating was on a scale of 1 to 5 where 1 was strongly disagree and 5 was strongly agree 



Item and (n) 


Mean/Mode 
Rating Before 
Project 


Mean Rating 
After Project 


Mean Change 


I use the standards as a basis for planning 


2.3 


3.7 


1.4* 


learning activities. (n=88) 


2.0 


4.0 




When planning or selecting learning 


3.0 


3.7 


.7* 


activities, I focus on what and how well 


3.0 


4.0 




students are learning rather than on how 








engaged students are in an activity. (n=86) 









*p<.0001 



The interview contained questions related to changes in how often teachers used standards- 
based practices in the classroom. A downside of our selection criteria for the interview 
sample was that we often got teachers who had been involved in standards before the 
project. This made it difficult to identify project effects from the effects of other standards- 
related interventions. We were surprised to learn that they defined the IAS project as using 
the software only and not also focusing on writing activities linked to the standards. From 
the teacher perspective, however, if they were already involved in standards implementation, 
the software was the most distinct characteristic of the IAS project. We needed to probe to 
get clarification about whether, aside from using the software, or not the process to develop 
learning activities in the project was different than how they would usually plan instruction. 
Once this was clarified, we learned the following: 

• Of the twenty teachers interviewed, three fourths said the process for developing 
learning activities in the project was a lot or somewhat different that what they would 
usually do. The majority (11 people) described the difference as thinking of 
standards first and making a link between standards and activities. Three people said 
the process was different because they were writing down activities and provide 
more detail. 

• One fourth of the teachers said the process was not at all different. All of these 
teachers had prior involvement with the standards. 

A majority of the teachers who were interviewed (15) said they were very or somewhat likely 
to continue using the process developed in the project to create learning activities. The five 
teachers who answered “very unlikely” said that the software was the barrier. Answers to 
the above questions are hard to interpret because some schools wrote activities without 
software and some only with software so they had different frames of reference for the term 
“process.” To the five who answered very unlikely because the software was a barrier, they 
considered the process using the software, not thinking about results first. 
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Effect on Instruction and Assessment Practices 

In addition to measuring change in the process teachers use to develop activities, hopefully 
to looking at the standards first, the survey also asked teachers to describe their agreement 
with items describing standards-based practices as reflective of their practice before the 
project and currently (see Table 8). A higher number on the current rating indicates that the 
teacher’s practice is more in agreement with the statement. The difference for each teacher’s 
two ratings was calculated to assess the degree of change. Both items showed significant 
positive change as follows: 

• I use the standards to explain expectations to students, (average increase of 1.3 
points on a five-point scale). 

• I link assessments to standards, (average increase of 1.3 points on a five-point scale). 
Each of these changes was statistically significant on a matched pair’s t-test. 



Table 8: Change in Use of Standards for Instruction and Assessment 



Item and (n) 


Mean/Mode 
Rating Before 
Project 


Mean Rating 
After Project 


Mean Change 


I use the standards to explain expectations 


2.2 


3.5 


1.3* 


to students. (n=88) 


1.0 


4.0 






2.5 


3.8 


1.3* 


I link assessments to standards. (n=86) 


2.0 


4.0 





♦pC.0001 



To supplement the information gathered through the survey, the interview asked about 
whether the project changed how teachers instruct and assess students. A majority (1 1) of 
the teachers who were interviewed said the project made little or no change in how they 
instructed. In most cases, they said this was because they had already been instructing 
consistent with the standards or because they had been involved in other standards 
implementation efforts and it was difficult to sort out the impact of each project on their 
instruction. 

Seven teachers said the project changed how they instruct some or a lot. Three said that 
now they know where they want their students to go for every lesson. One talked about 
changes in grading practices - now he could set expectations for the minimum amount of 
work a student needed to do to receive credit - , another talked about seeing more student- 
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directed learning, another about more project-based learning. In addition, one of the 
teac ers descnbed why she thought the project made standards more accepted by teachers: 

First came the standards and they were theoretical. Then, at our school, IAS 
focused on developmental activities; activities students would do to prepare for the 
standards. I think finally teachers saw how what they do in the classroom fits. 
Instead of big [assessment] packages they were working with learning activities and 
this made them more comfortable. There was less change required.” 

Two teachers did not rate the level of change. 

Based on the interviews, the project was more likely to effect how teachers assess their 
students. A majority (1 1) of teachers said their involvement in IAS changed how they assess 
students a lot or some. Five said they were using performance packages and thus their 
assessments were more authentic. Three mentioned the use of checklists to show students 
what is expected for a passmg grade. Two mentioned that they now involved students more 
in assessing their work; students may help develop rubrics to assess performance or give 
mput mto how they think they are doing on an assessment. 

Of the 9 teachers who said the project had litde or no affect on how they assess students, 
five said they were already using a lot of performance assessment and another said the 
change was due to the graduation standards not the project. 



Summary and Conclusions 

The focus of this study was a project (IAS) designed to develop and pilot-test a process for 
implementing Minnesota’s high standards for student learning. The state department of 
education (MDE) has mandated basic standards and high standards for Minnesota students. 
Current state rules require students to achieve passing scores on state-developed basic skill 
tests m reading, mathematics, and writing in order to graduate from high school. 

The assessment of the high standards, however, provides school districts with much more 
flexibility. Because state law forbids a mandated state curriculum, each district must define 
its own instructional plans, and districts can also modify the state-developed assessment 
packages to fit their local context. Therefore, IAS leaders believed the next step in 
implementing the standards was to have teachers develop learning activities that could be 
used to prepare students for the assessments. In addition to developing a process for 
teachers to use in creating standards-based learning activities, the project also attempted to 
establish a network of teachers who were implementing the standards and to create an 

electronic library of learning activities linked to the standards. 

We were hired by IAS to document their development of a process for integrating 
Minnesota s high standards for student learning into classroom instruction, to assess the 
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effectiveness of this process, and to identify effective practices for building teacher capacity 
to implement standards. Our study attempted to answer the following questions: 

What process did IAS develop for integrating Minnesota’s high standards into 
classroom instruction? What did IAS learn about effective practice for integrating 
high standards into instruction? 

To what extent does the process IAS developed increase teacher capacity to 
implement Minnesota’s high standards for student learning? 

To what extent did IAS develop a network of teachers who are working to 
implement Minnesota’s high standards for student learning? 

To what extent did IAS increase the level of standards-based practice among 
teachers who participated in the project? 



In this section we summarize the results of our study and discuss the implications for further 
efforts to implement standards-based reforms. VC e also discuss areas where further research 
is needed to increase our understanding of the change theory behind standards-based reform 
and to increase the chance that it will ultimately improve student achievement. 



A Process for Developing Standards-Based Instruction 

Perhaps the most significant contributions of this project to our understanding of howto 
implement standards-based reform were the identification of a district-level role for 
implementation, and the creation of a process for developing a standards-based curriculum. 
This is consistent with the finding of Massell, Kirst, and Hoppe ( 1997 ) that policymakers 
have overlooked an important role for districts in carrying out standards-based reform. This 
project identified a role for the district and a process, sketching the standards, for creating an 
integrated, district-wide plan for providing standards-based instruction and assessment. 

The project also recognized the importance of helping teachers recognize the difference 
between learning activities and performance tasks. Working with a specialist in curriculum 
and assessment, the LT decided that the purpose of learning activities is to provide students 
with instruction and practice on skills they will need to demonstrate later on a performance 
task. Therefore, the checklist, or list of what a quality performance is, in a learning activity 
should not be graded but only used to communicate expectations to students and for 
student self-evaluation. 



Increase Teacher Capacity to Implement Standards 

Teacher’s self-reports on the survey indicated positive changes in their knowledge and 
beliefs related to the standards. The changes were statistically significant on all but one item. 
This suggests that the process IAS developed for teachers to write standards-based 
instructional activities was effective in increasing their understanding of how to use 
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standards in instruction, and in making their attitudes toward standards more positive. 
However, given the low response rate to the survey we are hesitant to generalize these 
results to all teachers. 

Teachers’ knowledge and skills are one part of Newmann, King, and Rigdon’s (1997) 
organizational capacity. They believe external accountability systems, such as the Minnesota 
standards, will not be effective in improving student learning unless schools develop their 
organizational capacity. The results of this study suggest that by developing standards-based 
instructional activities for their own students teachers can increase their skills and knowledge 
related to implementing standards-based education. 

The relationship between participation in IAS and the observed changes in teacher’s 
knowledge and beliefs can not be determined definitively from this study. Many of the 
teachers in this study were involved in other standards-related initiatives simultaneously with 
their participation in IAS, making it difficult to attribute any changes specifically to IAS 
activities. Further research is needed using a design that incorporates a comparison group in 
order to better assess the specific effects of the IAS process on teacher knowledge and 
beliefs. 

Further development of tools to measure changes in teacher knowledge and skills related to 
the standards is also needed. Part of the difficulty in designing such tools is the lack of a 
consistent definition of what skills and knowledge teachers need to implement standards. In 
a specific context, however, researchers could assess the level of teacher skill by rating the 
quality of the activities they develop, assuming that the teachers and administrators in that 
context could agree on measurable indicators of quality. 



Build a Network of Teachers Implementing Standards 

A key part of the project’s theory of change was that teachers would work together to 
develop instructional activities and build a network of teachers that could support each other 
in using these activities in the classroom and assessing students. 

For most of the teachers in the project, the strategies IAS used to encourage collaboration 
were not effective. Only 8% of the teachers in the project always worked with others to 
write activities. Almost half (47%) reported that they always worked alone or primarily 
worked alone. 

Of the teachers who reported working with others at least some of the time, half spent less 
than four hours collaborating with colleagues to develop learning activities. The other half 
spent anywhere from five to 200 hours collaborating, and the most typical total amount of 
time they reported collaborating during the project was nine hours. Only 3% of the teachers 
used e-mail to collaborate; the primary vehicle for communication was in-person meetings. 

One explanation for the low level of collaboration may be the lack of time. Many of the 
teachers interviewed said that it was difficult to find time to meet with colleagues during the 
school year. Hence, they either developed activities completely on their own or only sought 
feedback from colleagues once the activity was fully developed. Further efforts at standards- 




30 



28 



> 1 I - 



implementation should allocate more funding for curriculum development time so teachers 
can, perhaps, find more time to collaborate. 

The study provides preliminary evidence that working together to develop standards-based 
instruction may help teachers develop a shared commitment, another component of 
Newmann, King, and Rigdon s (1997) organizational capacity. Teachers who worked with 
others at least some of the time were most likely to establish on agreement in the area of 
what students should learn in school. This agreement can enhance student learning because, 
as students progress through the school system, they are more likely to encounter teachers 
that have a shared vision of what students should be learning. 

Due to complications at several points in the project’s implementation plan, this study does 
not provide an adequate test of how effective of the project’s strategies to encourage 
collaboration (common activity template, e-mail, electronic libraries of activities) may be. 

For example, the inability of teachers to access the electronic library during the project 
meant that we could not assess how often teachers used the library or whether they found it 
helpful in developing their own instruction. Most sites were not able to provide teachers 
with reliable access to e-mail during much of the project, a factor that likely reduced teacher 
use of this method for communicating with other developers. 



Increase the L evel of Standards-Based Instruction and Assessment 

Items on the survey related to teacher use of standards in developing instruction, providing 
instruction, and assessing students showed that teachers noted a significant increase in their 
level of use during the project. The item that showed the largest magnitude of positive 
change was “I use the standards as the basis for planning learning activities.” Another 
change was that teachers increased their use of the standards to explain expectations to 
students. 



In the sample of teachers who were interviewed, three-fourths said the process for 
developing learning activities in IAS was different than what they would usually do. The 
majority described the difference as thinking of standards first and making a link between 
standards and activities. 

As with the reported changes in teacher knowledge and beliefs about the standards, the 
relationship between IAS and changes in teacher’s use of standards-related practices can not 
be determined from this study. While they were involved in IAS many teachers were 
involved in other projects related to standards, some of more intensity than IAS. 

Further development of tools to measure standards-related practices is needed. In addition 
to teacher self-report it would be important to incorporate observations by an external 
researcher to avoid the potential of self-report bias. Further research is needed in a situation 
where a comparison group is available so we could better isolate the effects of IAS’s process 
for implementing standards from other district initiatives. 
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